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ABSTRACT 

We  consider  the  problem  of  dynamic  scheduling  of  customers  (messages)  in  time-critical  envi¬ 
ronments.  First,  we  consider  a  single  station  (communication  node)  and  assume  that  each  customer 
(message)  must  begin  service  (transmission)  by  an  individually  varying  “extinction”  time  or,  else, 
it  is  lost.  We  are  interested  in  minimizing,  in  the  sense  of  stochastic  order,  the  number  of  mes¬ 
sages  lost  over  any  time  interval.  We  prove  a  variety  of  results  that  establish  the  optimality  of  the 
STE  (Shortest-Time-to  Extinction)  policy  under  rather  general  conditions.  Similar  results  are  also 
shown  when  messages  have  constraints  on  their  complete  transmission  times.  If  the  scheduler  is 
allowed  to  take  decisions  based  only  on  the  distribution  of  the  deadlines  (rather  than  their  exact 
values),  similar  but  somewhat  stronger  results  are  proven.  Finally,  we  consider  a  network  of  M  sta¬ 
tions  in  tandem  under  the  hypothesis  that  a  message  is  never  lost  and  is  scheduled  irrespective  of 
whether  its  extinction  time  (also  called  due  date  in  this  case)  has  expired  or  not.  Again, under  fairly 
general  assumptions  on  the  arrivals,  deadlines  and  services,  we  show  that  the  EDD  (Earliest  Due 
Date)  policy  minimizes  a  form  of  average  tardiness  incurred  over  a  finite  operating  horizon  among 
all  nonidiing,  nonpremptive  policies.  We  formulate  these  problems  in  the  context  of  stochastic 
dominance,  and  use  simple  interchange  arguments  to  establish  all  our  results. 


*  This  work  was  supported  by  ON R  Grants  N00014-85G-0207  and  N00014-88K-0712  and  an  N SF  Grant 
CDR-85-00108. 
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I.  INTRODUCTION 


We  consider  the  problem  of  scheduling  the  transmission  of  messages  over  a  single  communi¬ 
cation  link  when  each  message  has  constraints  on  its  waiting  time  or  complete  transmission  time. 
This  problem  arises  in  applications  that  involve  time-critical  message  contents.  We  wish  to  model 
situations  in  which  the  penalty  incurred  when  deadlines  are  not  met  implies  either  the  complete  loss 
of  the  message  or  another  form  of  tardiness  cost.  We  are  interested  in  characterizing  the  scheduling 
strategy  which  minimizes  a  cost  function  that  reflects  the  nature  of  the  penalty  incurred.  The 
models  and  the  results  of  the  paper  apply  equally  well  to  numerous  other  applications  that  involve 
service  stations,  queues  and  deadlines.  Thus  a  much  more  general  terminology  could  be  used. 
We  choose  to  stick  with  the  message  transmission  application  in  order  to  focus  attention  to  the 
important  problem  of  real  time  communication. 

First  we  consider  the  case  in  which  the  messages  have  constraints  on  their  waiting  times.  Each 
message  upon  its  arrival  at  time  “announces”  a  deadline  d,-,  so  that  if  by  time  e*  =  t,  +  d,  (called 
its  “extinction  time”),  transmission  does  not  commence ,  the  message  is  considered  lost  and  never 
scheduled  for  service.  The  objective  is  to  find  a  scheduling  policy  which  minimizes  the  average 
number  of  lost  messages  over  any  time  interval.  We  show  that  under  nonexplosive,  but  otherwise 
arbitrary,  arrival  and  arbitrary  deadline  processes,  and  for  exponential  service  (i.e.  transmission) 
times  that  are  independent  of  each  other  and  of  the  arrival  and  deadline  processes,  the  policy  of 
scheduling  the  eligible  customer  with  the  Shortest  Time  to  Extinction  (denoted  by  STE)  is  optimal 
among  all  nonpreemptive  and  nonidling  policies.  In  fact,  we  show  the  optimality  in  the  sense  of 
stochastic  order.  When  considered  over  the  broader  class  of  only  nonpremptive  policies,  the  optimal 
policy,  if  it  exists,  can  be  found  in  the  class  of  STEI  policies,  namely  those  that  are  allowed  to  idle, 
but  schedule  according  to  the  STE  rule  when  they  don’t  idle.  As  a  special  case,  in  the  situations 
in  which  deadlines  are  deterministic  and  identical  for  all  messages,  the  pure  STE  policy  is  optimal 
within  the  class  of  nonpreemptive  policies. 

Next,  we  consider  the  case  in  which  messages  have  constraints  on  their  complete  transmis¬ 
sion  times  rather  than  on  their  waiting  times  prior  to  service  commencement.  A  message  is  now 
considered  lost  if  it  does  not  complete  transmission  by  its  extinction  time.  We  assume  that  the 
transmission  of  a  message  is  aborted  if  its  deadline  expires  while  it  is  in  the  process  of  being  trans¬ 
mitted.  Scheduling  results  that  are  similar  to  those  of  the  previous  case,  are  also  obtained  for  this 
case.  Preliminary  versions  of  these  results  were  first  presented  in  [15]. 

An  interesting  twist  to  the  above  problem  is  obtained  if  we  assume  that  due  to  implementational 
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difficulties,  the  scheduler  doesn’t  have  information  about  the  exact  deadlines  of  the  messages.  In 
addition  to  the  knowledge  of  the  past  evolution  of  the  system,  only  the  knowledge  of  the  distribution 
of  the  deadlines  is  available  for  decision  making.  However  the  extinction  times  become  known  as 
a  message  is  read  and  the  scheduler  can  act  as  before.  We  assume  that  the  deadlines  form  a 
sequence  of  i.i.d  RV’s  (that  are  also  independent  of  the  arrivals),  and  the  common  distribution  has 
non-decreasing  failure  rate.  Service  times  are  also  assumed  to  form  an  i.i.d  sequence  of  RV’s  that 
are  independent  of  the  arrivals  and  deadlines.  When  deadlines  are  to  the  beginning  of  service,  the 
policy  of  scheduling  the  message  which  has  waited  the  most,  is  shown  to  be  optimal  within  the  class 
of  non- preemptive  policies.  Exponentiality  of  service  times  is  needed  further  when  the  deadlines 
are  to  the  completion  of  service. 

These  results  do  not  seem  to  be  easily  extendible  beyond  the  case  of  a  single  link.  However, 
under  a  slightly  different  set  of  assumptions  and  operating  conditions,  some  results  can  be  obtained 
for  a  tandem  network  of  links.  Specifically,  we  may  assume  that  no  messages  are  discarded  or  lost 
and  instead,  all  messages  are  scheduled,  regardless  of  whether  their  deadlines  have  expired  or  not. 
A  penalty  is,  however,  incurred  when  a  deadline  is  missed.  The  penalty  function  is  of  the  form 
h(ci  -  e;)  where  Cj  is  the  time  at  which  the  message  arriving  at  t{  with  deadline  d,  (and  extinction 
time  e*,  which  in  this  case  is  also  called  due  time)  completes  transmission  in  the  network,  and  h 
is  a  real  valued  continuous  convex  function  with  h(x)  =  0  Vx  <  0.  For  this  system,  we  consider 
a  finite  operating  time  horizon  and  wish  to  obtain  a  scheduling  policy  that  minimizes  the  total 
penalty  function  (usually  called  tardiness).  Under  nonexplosive,  but  otherwise  arbitrary,  arrival 
and  arbitrary  deadline  processes,  and  independent  identically  distributed  service  times  that  are 
also  independent  of  the  arrival  and  deadline  processes,  we  show  that  the  policy  which  schedules, 
at  each  node,  the  message  with  the  Earliest  Due  Time  is  optimal  among  all  non-preemptive  and 
non-idling  scheduling  policies. 

These  problems  fall  in  the  category  of  single  server  queueing  systems  with  impatient  customers. 
Such  systems  have  received  moderate  attention  in  the  queueing  literature.  Most  of  the  work  seems 
to  have  focussed  on  the  evaluation  of  various  criteria  of  performance  when  service  is  assigned 
according  to  a  First-come-first-serve  (FCFS)  strategy.  For  example  in  [1,2],  the  authors  are  able 
to  compute  various  performance  indices  such  as  steady  state  probability  of  rejection  of  a  customer, 
average  number  of  customers  served  before  the  loss  of  the  first  job  etc.  for  a  FCFS  single  server 
queue.  In  [3],  delay  analysis  is  given  for  a  single  server  queue  with  an  interesting  delay- dependent 
service  discipline;  this  models  telephone  call  processing  systems. 
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While  intuition  suggests  that  assigning  service  according  to  a  FCFS  discipline  is  probably  not 
the  best  thing  to  do,  attempts  to  discover  better  scheduling  stragies  have  been  rare.  In  [4,5],  the 
problem  of  optimally  scheduling  the  service  of  impatient  customers  was  posed  and  weaker  versions  of 
some  of  our  results  were  proved  with  a  different  technique  and  under  different  assumptions  relative 
to  ours.  In  [6],  the  problem  of  optimal  control  of  arrivals  to  a  FCFS  single-server  queue  was 
considered  with  the  objective  of  minimizing  the  discounted  reward  associated  with  the  successful 
departure  of  customers.  An  optimal  control  problem  with  interacting  service  stations  and  impatient 
tasks  was  studied  in  [16].  The  authors  assumed  that  the  extinction  times  are  not  known  to  the 
controller  and  the  residual  time  (until  extinction)  is  exponential.  The  service  assignment  strategy 
minimizing  a  (discounted)  weighted  average  delay  was  characterized.  Several  researchers  [7,8,9]  in 
the  operations  research  area  have  considered  scheduling  impatient  customers  under  various  tardiness 
criteria.  In  contrast  to  our  formulation,  these  works  assume  that  the  arrival  process  is  shut  off  after 
the  system  has  started  the  scheduling  operations  and  prove  results  only  for  a  single  node. 

The  nature  of  the  problem  we  consider  makes  it  difficult  to  formulate  a  dynamic  programming 
recursion.  Our  approach,  instead,  involves  interchange  arguments  together  with  the  ideas  of  cou¬ 
pling  and  stochastic  dominance  and  is  based  on  sample-path  wise  comparisons  of  the  costs  under 
different  policies.  Somewhat  similar  arguments  were  given  in  [10,11]  for  queue-control  problems  of 
a  different  nature.  However,  since  results  regarding  the  existence  of  optimal  stationary  policies  are 
not  readily  available  for  our  problem,  some  additional  care  is  needed. 

The  paper  is  organized  as  follows.  In  section  II,  we  introduce  the  notation  and  consider  the  first 
situation  in  which  messages  have  constraints  on  their  waiting  times.  The  case  in  which  constraints 
are  placed  on  their  total  transmission  times  is  considered  in  section  III.  In  section  IV,  we  analyze 
optimality  under  the  reduced  information  structure.  Finally  in  section  V,  we  present  the  case  of 
the  tandem  network  with  tardiness  cost. 

II.  CONSTRAINTS  ON  WAITING  TIMES 

We  consider  a  single  server  queueing  system  with  unlimited  buffer  space  size  that  represents 
a  single  link  of  a  communication  system.  Let  t{  be  the  arrival  instant  of  the  ith  message  whose 
deadline  is  d;.  We  define  by  e;  =  f;  +  dj  the  extinction  time  of  that  message;  that  is,  if  by  time 
e,,  the  transmission  does  not  commence,  the  message  is  considered  lost  and  never  scheduled  for 
service.  At  any  instant  t,  a  message  with  extinction  time  e;  is  termed  eligible  for  transmission  if 
e*  —  t  >  0.  Let  {T{  =  —  fi_i}  •’[11  ( with  to  =  0 )  be  the  sequence  of  interarrival  times  and  Si  be 
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the  duration  of  the  ith  service  time  in  the  system.  By  service  time,  we  mean  the  transmission  time 
of  a  message  which  may  include  processing  and  propagation  times  as  well. 

We  make  the  following  assumption  throught  the  paper  : 

(Al)  {5,} is  a  sequence  of  independent  identically  distributed  RV’s  which  are  independent  of 
and  Also,  the  arrival  process  is  nonexplosive,  that  is,  limjfootj  =  oo  with 

probability  1. 

Let  E(t)  denote  the  (increasing)  ordered  set  of  extinction  times  of  eligible  messages  at  time 
t.  Let  Ha(t)  denote  the  set  of  all  arrival  instants  by  time  t  and  Ed(t)  the  set  of  corresponding 
deadlines.  Also,  let  C,{t)  be  the  expended  portion  of  the  service  in  progress  at  time  t  and  i(t) 
be  the  condition  of  the  server  at  time  t  (1  if  busy,  0  if  idle).  Then,  under  assumption  Al,  z(t)  = 
(E(t),Ha(t),Hd(t),C3(t),i(t))  is  a  useful  description  of  the  system.  Let  Z  denote  the  allowable 
range  of  values  of  z{t). 

The  control  action  is  to  decide,  at  appropriate  decision  instants,  whether  to  transmit  and,  if 
so,  which  message  out  of  the  currently  eligible  pool  of  messages.  First,  we  restrict  attention  to 
nonanticipative  policies  throughout  this  paper.  This  means  that  the  control  action  has  to  be  based 
only  on  the  past  evolution  of  the  system,  specifically,  the  knowledge  of  the  service  times  of  the 
messages  waiting  in  the  queue  is  not  available.  Let  To  be  the  class  of  nonpreemptive  and  nonidling 
policies  and  Tx  be  the  class  of  nonpreemptive  policies,  while  T  is  the  global  class  of  all  possible 
scheduling  policies. 

For  every  policy  in  To,  the  decision  instants  are  the  instants  of  service  completion  (provided 
that  E(t)  at  these  instants  is  nonempty)  or  of  arrivals  to  an  empty  queue.  Denote  by  STE,  the 
policy  in  To  which  at  every  decision  instant  schedules  the  eligible  message  with  the  shortest  time 
to  extinction.  Let  STEI  denote  the  class  of  policies  in  Tx  which  are  allowed  to  possibly  idle  when 
messages  are  waiting  in  the  queue,  but  which  schedule  according  to  the  STE  mechanism  when  they 
choose  not  to  idle. 

Following  the  standard  notation,  we  say  that  a  RV  X  is  stochastically  smaller  than  a  RV  Y, 
and  write  X  <,<  Y  if  P(X  >  z)  <  P(Y  >  z)  Vz  6  M.  Order  relations  for  stochastic  processes  can 
be  considered  as  an  extension  of  the  definitions  for  vector  valued  RV’s.  Let  X  =  {X(t),t  6  A} 

and  Y  =  {Y(i),f  €  A}  be  two  processes,  where  A  C  M.  Let  D  d=  X>h[0,  oo),  the  space  of  right 
continuous  functions  from  2R+  to  M  with  left  limits  at  all  t  e  [0,  oo)  be  the  space  of  their  sample 
paths.  We  say  that  the  process  X  is  stochastically  smaller  than  the  process  Y,  and  write  X  <st  Y 
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if  P{f(X)  >  z)  <  P{f(Y)  >  z)  iz  6  1R,  where  /:  D  »-►  JR  is  measurable  and  /(*)  <  /(y)  whenever 
x,y  €  D  and  x(t)  <  y(t)  it  G  A.  The  following  equivalence  [12,13]  often  provides  an  easy  way  to 
prove  stochastic  order  relations  without  explicit  computation  of  distributions: 

1. X  <at  Y 

2.  P(g[X(ti),  ■  •  •,  A"(t„)]  >  z)  <  P(g[Y(t1),-.-,Y(tn)])  >  z)  for  all  (tu  •  •  all  2,  all  n,  and  for 
all  g:  Mn  M,  measurable  and  such  that  Xj  <  yj,  1  <  j  <  n  implies  g(x\,.  •  •,»„)  <  y(yj,  •  •  -,y„). 

3.  There  exists  two  stochastic  processes  X  =  {X(f),f  €  A}  and  T  =  {y(t),t  £  A}  on  a  common 
probability  space  such  that  C{X)  =  C(X)  ,  C(Y)  =  C(Y)  and  X(t)  <  Y(t)  it  6  A  a.s.  Here  £(.) 
denotes  the  law  of  a  process  on  the  space  of  its  sample  paths. 

Returning  to  our  problem,  let  iir(^)  denote  the  process  {LJ(z),t  >  0},  where  LJ(z)  is  the 
number  of  messages  lost  by  time  t  when  starting  from  state  z  at  time  0  and  applying  the  scheduling 
policy  jt. 

We  first  consider  optimality  within  To. 

Theorem  II.  1.  Consider  a  single  server  queue  under  assumption  Al.  Assume  further  that  the 
common  distribution  of  the  service  times  is  exponential.  Then,  the  STE  policy  minimizes  in  the 
sense  of  stochastic  order,  the  number  of  messages  lost  by  any  time  among  all  policies  in  the  class 
To,  that  is, 

Lste{z)  <tt  L*{z)  in  €  r0,Vz  6  Z. 

Under  the  assumption  of  Poisson  arrivals  rather  than  exponential  services,  STE  policy  was 
shown  in  [5]  to  maximize  the  long  run  expected  fraction  of  successful  messages  among  all  stationary 
policies.  Using  the  fact  that  the  initiation  of  busy  periods  are  regeneration  points,  the  conclusion 
followed  from  a  comparison  of  the  quantities  for  the  first  busy  period.  Here,  we  prove  a  stronger 
result  using  different  techniques.  We  first  need  the  following  result: 

Lemma  II.l.  Consider  a  single  server  queue  as  in  Theorem  II.l.  Let  an  arbitrary  policy  x  G  T0  act 
on  the  system  in  [to,  oo),  where  to  is  an  arbitrary  decision  instant.  Then  there  exists  a  policy  ft  €  T0 
that  schedules  the  customer  with  the  shortest  time  to  extinction  at  time  t0  (and  is  appropriately 
defined  in  [to,  oo))  and  satisfies 

L*(z)  <tt  L*(z)  iz  G  Z. 
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Proof.  Assume  that  r  does  not  schedule  the  customer  with  the  shortest  time  to  extinction  at  time 
to.  If  it  does,  the  result  follows  trivially  by  letting  if  to  be  the  same  as  7 r.  We  drop  z  from  LJ(z) 
and  L*(z)  for  notational  convenience. 

The  idea  of  the  proof  is  to  define  #  appropriately  in  [to,  00)  and  to  construct  two  coupled  pro¬ 
cesses  (Zf,  Lf)  on  the  same  (given)  probability  space  so  that  L*  and  Zf  have  the  same  distribution 
and  Zf  <  Zf  a.s'it  >  to- 

Suppose  E(to)  —  {ei,  •  •  -,en)  with  n  >  2.  We  agree  to  denote  by  e<  either  the  extinction  time 
or  the  message  with  that  extinction  time.  Let  7 r  schedule  ek  (2  <  k  <  n)  time  to.  We  will 
construct  ft  6  To  which  schedules  e^-i  at  time  to  (  and  is  appropriately  defined  in  [fo,oo))and 
satisfies  the  assertion  of  the  lemma.  The  required  policy  tt  can  then  be  generated  by  induction  on 
k. 

Consider  the  system  evolving  under  policies  if  and  7r.  Couple  the  realizations  by  giving  them  the 
same  arrival  and  deadline  processes  in  [to,  00).  Let  a  be  the  completion  instant  of  the  service  which 
begins  at  time  to  under  ft.  Take  the  service  under  tt  to  end  at  a  as  well.  This  is  permissible  since 
the  service  times  are  independent  and  identically  distributed.  Three  cases  exhaust  the  possibilities: 
Case  1  :  0  >  ek 

In  this  case,  under  both  policies,  all  messages  eligible  at  time  t0,  except  ek  for  w  and  efc_i 
for  ft  that  have  extinction  times  less  than  or  equal  to  a  are  lost  and  so  are  all  arrivals  in  (to,  cr] 
whose  extinction  times  are  similarly  less  than  or  equal  to  a.  The  states  under  7r  and  ft  are  therefore 
matched  at  time  a.  In  [<7, 00),  define  ft  to  be  identical  to  tt;  this  is  possible  since  we  may  take 
corresponding  service  times  to  be  equal  under  tt  and  if.  Thus, 


f  Zf  iff  €  [*o»e*-i)U[e*,oo); 
1  Zf  +  1  if  f  e  |ejt_i,efc). 


Case  2  :  a  <  ek-x 

First  it  is  clear  that  Zf  =  Zf  Vf  G  [to,<^),  and  at  time  <7,  the  sets  of  extinction  times  under 
7 r  and  ft  differ  only  in  that  ek~i  is  included  in  that  set  under  7r  as  compared  to  ek  under  if.  Let 
if  follow  7r  for  t  >  cr  except  that  it  schedules  ek  when  (and  if)  tt  schedules  ek-i.  Thus  ft  is  well 
defined  in  [cr,  r),  where  r  is  the  end  of  the  current  busy  period  under  7T. 

Suppose  7 r  eventually  schedules  ek-i,  that  is,  suppose  that  ek-\  meets  its  deadline  under  7r. 
Since  ek-i  <  ek,  ek  will  also  meet  its  deadline  under  ft ;  and  the  states  under  tt  and  ft  are  matched 
at  time  r.  Letting  if  follow  x  in  [r,  00),  one  thus  obtains,  Zf  =  Z^  Vt  €  [a,  00). 

Suppose  now  that  tt  does  not  manage  to  eventually  schedule  ek-i  before  its  expiration.  Since 
7r  is  nonidling,  we  have  necessarily  that  r  >  e^-i.  If  r  >  ek,  then  ek  is  lost  under  if.  Letting  ft 
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follow  7 r  in  (r,oo),  one  obtains, 


F*  fXf  if*  6  [ff.eMjUje^oo); 

*  1  L*  +  1  if  t  G  [e*-i,«fc). 

If  however,  r  <  e*,  then  at  time  r,  the  queue  is  empty  under  7r  and  ek  is  the  only  eligible  message 
under  7 r.  Let  x  begin  serving  e*  at  time  r.  We  must  now  consider  the  following  two  sub-cases: 

(a)  Suppose  there  are  no  arrivals  while  ek  is  in  service  under  x.  The  states  are  then  matched  from 
the  instant  message  ek  finishes  service  under  i r;  thus, 


/  L*t  if  t  G  [<r,e*_i); 
\Xf  +  l  if  t  6  [efc_i,oo). 


(b)  Suppose  there  is  at  least  one  arrival  while  e k  is  in  service  under  x.  Let  the  arrival  which  begins 
service  first  under  x  have  extinction  time  e*,.  Take  the  service  time  of  tb  under  7r  to  be  equal  to  the 
residual  service  time  of  ek  under  it.  This  is  possible  because  of  assumption  A1  and  the  memoryless 
property  of  the  exponentially  distributed  service  times.  In  this  way,  we  ensure  that  ej,  and  ek  will 
finish  service  at  the  same  time  instant  <T\.  Suppose  that  <J\  >  e^.  Then  is  lost  under  it  and  the 
states  under  the  two  policies  are  matched  at  time  <T\.  One  thus  concludes  that, 


Fir  _  [  Xf  if  <  €  [(7,  efc_i)  U  [e;,,  oo); 

1  \x?  +  l  if  te[ek-ueb). 

Suppose  now  that  o\  <  e^.  Let  x  follow  x  for  t  >  a\.  Take  the  corresponding  service  times  under 
x  and  x  to  be  equal.  Thus  x  is  well  defined  in  [o\ ,  t\  )  where  t\  is  the  end  of  this  busy  period  under 
x.  If  7"i  >  e&,  we  let  x  follow  x  from  time  7"i  onwards,  and  thus  have,  the  same  relations  between 
Lj  and  LJ  as  in  the  situation  just  described.  If  instead  ti  <  eb,  let  x  begin  serving  e*,  at  time  rj. 
We  are  now  back  to  a  situation  we  have  described  previously.  One  easily  repeats  the  arguments  to 
obtain, 

Fir  _  f  Xf ,  if  f  G 

**  "Uf  +  1,  if «  €  [c*_x,n); 


and  Xf  >  X *  Vt  S  [7-1,00). 

Case  3  :  e*_i  <  a  <  ek. 

It  is  clear  that 

ji r  _  /  Xf,  lit  e[t0,ek-i); 
1  \Xf  +  l,  if  t  G  [ek-i,cr), 
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and  at  time  <r,  under  if,  message  e*  is  eligible  for  service  in  addition  to  all  messages  that  are  eligible 
under  x.  Consequently,  we  can  proceed  as  in  case  2.  We  thus  conclude  that 

L*t>L*t  Vt€[«r,  oo). 

The  observation  that  the  processes  L* and  LT  have  the  same  law  now  completes  the  proof. 

We  now  proceed  to  the 

Proof  of  Theorem  II.l.  Start  with  an  arbitrary  policy  7r  €  To  acting  on  the  system  in  an  initial 
state  z.  By  Lemma  II.l,  we  can  construct  an  alternative  policy  7Tx  €  To  which  schedules  according 
to  the  STE-rule  at  the  first  decision  instant  along  its  trajectory  and  which  satisfies  the  relation 
L **  <„t  L*.  We  proceed  inductively;  that  is,  by  repeating  the  same  construction  n  times  we  can 
define  a  policy  trn  6  To  which  schedules  according  to  the  STE-rule  at  least  at  the  first  n  decision 
points  along  its  trajectory  and  satisfies 

L*»  <st  L*'-1  <>t  ••■L*1  <at  L\ 

Fix  x  e  IR,a  positive  integer*:  and  t,  G  [0,oo),l  <i<k  and  pick  g:Mk  h*  M.  Let  A 7  denote  the 
event  •  •  •  L]k)  >  x}  for  a  policy  7  €  F0.  Let  tj  =  ma Xi<i<kti  and  take  {5n}x°  to  be  the 

service  times  of  the  messages  in  the  system.  Since  the  policies  STE  and  xn  agree  on  their  first  n 
decisions,  one  has  for  all  n, 


P(A'U)  =  P(Aate  n  {]T  Si  >  tj})  +  P(Aate  n  Si  <  tj}) 

i= 1  t=l 

<  P(A’-  n  S,  >  tj})  +  p(£  Si  <  (,) 


t=l 


i=l 


<P(A^)  +  PC£Si<tj) 

i=i 

<p(a’)  +  p(£  ft  <tj). 

1=1 


Taking  the  limit  as  n  /*  00,  it  now  follows  that  P(Aate)  <  P{AT),  that  is,  Lste  <st  Lv. 
Remarks. 

1.  The  theorem  is  not  true  when  the  assumption  of  exponential  service  times  is  relaxed. 
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Counterexample: 

Consider  a  single  server  queueing  system  in  which  service  times  are  deterministic  and  identically 
equal  to  5  units,  and  there  are  exactly  5  arrivals  into  the  system  at  times  3,4, 12, 16,  and  16.5  units 
with  corresponding  deadlines  3,9,2, 9  and  3.5  units.  Let  a  message  with  extinction  time  3  units 
and  residual  service  time  5  units  be  initially  present  in  the  system.  Then  a  comparison  of  the 
Last-Come-First-Served  (LCFS)  and  STE  policies  shows 

E[Lltcfe]  =  L\c}a  =  l(t  >  6), 


E[L\te]  =  Latte  =  1  (t  >  14)  +  1(*  >  20), 
where  1(A)  is  the  indicator  function  of  the  event  A.  Thus, 

E[Lltcfa ]  <  E[L\te]  Vi  >  20. 

2.  A  close  look  at  the  proof  also  shows  that  the  exponentiality  of  the  service  times  was  used 
only  to  match  service  completions  of  arrivals  to  those  who  have  already  started  service  (e.g  case 
2(b)).  Also,  idling  doesn’t  pay  when  there  are  no  arrivals  into  the  system  after  it  has  started 
operation.  So,  if  we  consider  a  single  server  queue  under  assumption  A1  such  that  all  the  messages 
are  initially  present  in  the  system  and  there  are  no  additional  arrivals,  following  the  same  line  of 
reasoning  as  in  the  Theorem,  we  obtain  the  (stronger)  result 

Late(z)  <tt  L*(z)  VvtTuVzeZ. 

Next,  we  consider  optimality  within  Tj ,  the  broader  class  of  only  nonpreemptive  policies.  Now, 
the  idling  of  the  server  is  allowed.  In  this  case,  examples  can  be  easily  constructed  to  show  that 
the  STE  policy  is  no  longer  optimal;  the  basic  intuition  being  that  when  all  the  messages  awaiting 
service  have  large  extinction  times,  it  pays  to  idle  in  expectation  of  a  message  with  a  very  short 
deadline.  However,  the  philosophy  of  STE-scheduling  still  plays  an  important  role  as  the  following 
result  demonstrates. 

Proposition  II. 1  Consider  a  single  server  queue  under  assumption  Al.  Then,  for  every  policy 
■k  E  Ti,  there  exists  a  policy  w  E  ST  El,  such  that 

L*(z)  <st  L*(z)  Vz  E  Z. 
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As  in  the  previous  theorem,  the  proof  can  be  worked  out  in  two  steps.  In  the  first  step, 
assuming  that  to  is  arbitrary  decision  instant  at  which  x  schedules  a  message  that  is  not  the  one 
with  the  smallest  time  to  extinction,  we  construct,  using  the  knowledge  of  x,  a  policy  if,  which 
schedules  the  message  with  the  smallest  time  to  extinction  and  satisfies  L *  <at  L*.  This  then  can 
be  used  recursively  to  improve  upon  any  given  policy  in  Ti  until  the  improving  policy  belongs  to 
the  class  of  STEI  policies.  The  key  observation  which  in  fact  facilitates  the  arguments  in  the  first 
step  is  that  when  idling  is  permitted,  the  policy  x,  which  we  construct,  can  be  chosen  to  follow 
x  at  all  times  beyond  to.  Exponentiality  of  the  service  times  is  therefore  not  needed.  We  omit 
the  details.  Under  the  same  assumptions,  [5]  contains  a  different  proof  for  the  optimality  of  ST  El 
policies  for  long  run  expected  fraction  of  successful  customers. 

Consider  now  a  special  case  of  interest  that  involves  the  following  assumption: 

(A2)  The  deadlines  associated  with  the  messages  are  deterministic  and  identical  for  all  messages, 
that  is,  d{  =  d  for  all  i.  Also  the  messages  initially  present  in  the  system  all  have  extinction 
times  less  than  or  equal  to  d. 

As  a  simple  consequence  of  the  above  proposition,  one  now  has  the  following 

Corollary  II.  1.  Consider  a  single  server  queueing  system  under  assumptions  A1  and  A  2.  Then 
for  every  initial  state  z  £  Z,  we  have, 

Late(z)  <.t  L*{z)  Vjt  6  Tl 

Proof.  First  note  that  by  Proposition  II.l,  it  suffices  to  prove  the  claim  for  x  £  STEI.  We  proceed 
as  before.  Let  to  be  an  instant  at  which  a  policy  x  £  STEI  chooses  to  idle.  It  now  suffices  to 
construct  x  £  Tx  which  schedules  the  customer  with  the  shortest  time  to  extinction  at  time  to  and 
satisfies  L*  <tt  L* . 

Let  E(t0)  =  {ex  •  •  -en)  with  n  >  1.  Because  of  assumption  A2,  it  is  clear  that  the  arrivals 
in  (fo,oo)  have  extinction  times  no  smaller  than  en.  Let  r  be  the  first  instant  after  to  at  which  x 
decides  to  schedule  a  message.  Consider  the  following  two  cases  : 

1).  Suppose  t  <  e\.  Since  x  £  STEI ,  x  schedules  ex  at  time  r.  Take  the  first  service  time  under 
x  (which  begins  at  time  r)  to  be  the  same  as  the  service  time  of  ex  under  x. Following  its  own 
trajectory,  therefore,  x  can  determine  the  instant  of.  completion  of  service  of  ex  under  x.  Let  x  idle 
from  the  time  the  service  of  ex  is  finished  under  x  to  the  time  of  completion  of  service  of  ex  under 
x.  Letting  x  follow  x  from  this  time  onwards,  one  obtains  Lj  =  Lf  Vf  >  to¬ 
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2).  Suppose  t  >  e\.  Let  it  schedule  e*  at  time  r.  Take  the  service  time  of  e*  under  it  to  be  equal 
to  that  of  e\  under  it.  Let  it  never  schedule  e*  and  instead  idle,  if  necessary,  as  in  the  previous 
case.  It  is  then  clear  that 

jv  _  {  LJ  if  t  £  [t0,ei)  U  [e*,oo); 

4  Uf  +  1  if/6[ei,e*). 

Because  of  Assumption  Al,  Lv  and  L *  have  the  same  distribution,  and  so,  L *  <st  Lv. 

III.  CONSTRAINTS  ON  COMPLETE  TRANSMISSION  TIMES 

We  now  assume  that  messages  have  limitations  on  their  entire  sojourn  times  in  the  system. 
Each  message  upon  its  arrival  into  the  system  at  time  t,-  declares  a  deadline  d{  such  that,  if  by  its 
extinction  time  e,  =  U  +  di  its  transmission  is  not  completed,  the  message  is  considered  lost.  If  its 
extinction  time  occurs  while  it  is  awaiting  transmission,  it  is  never  scheduled.  It  may  also  happen 
that  the  message  is  in  the  process  of  being  transmitted  when  its  extinction  time  occurs.  We  assume 
in  this  case  that  its  transmission  is  aborted  at  that  moment  and  it  is  considered  lost. 

We  first  consider  optimality  within  the  class  of  nonpreemptive  nonidling  policies.  The  following 
parallels  Theorem  II.  1  closely.  We  provide  the  proof  in  its  entirety  but  in  a  somewhat  terser  fashion. 

Theorem  III.l  Consider  a  single  server  queue  under  assumption  Al  and  exponential  service  time 
distribution.  Then  for  every  initial  state  z  G  Z,  we  have, 

L'te(z)  <tt  L*(z)  Vtt  6  T0. 

Proof.  To  prove  the  result,  we  follow  the  same  program  as  illustrated  in  Theorem  II.l.  With 
E(to)  =  {ci  •  •  -en}  («  >  2),  let  it  €  To  be  a  policy  which  schedules  message  e*  (2  <  k  <  n)  at 
time  to-  Using  the  knowledge  of  it,  we  construct  f  which  schedules  e^-x  at  time  to  and  satisfies 
L*  <,t  L*.  First  an  induction  on  k  and  then  an  induction  on  the  decision  instants  will  provide  the 
final  result. 

Consider  the  systems  evolving  under  policies  it  and  it.  Couple  the  realizations  by  giving  them 
the  same  arrival  and  deadline  processes  in  [to,  oo).  Let  o  be  the  instant  at  which  the  service  which 
begins  at  time  to  under  f  would  complete  if  uninterrupted.  Take  the  service  time  of  e^  under  it  to 
be  equal  to  that  of  e^-i  under  it.  Two  cases  exhaust  the  possibilities: 

Case  1  :  a  >  e^-i. 

Consider  the  interval  [<o,efc_i].  All  arrivals  in  this  interval  having  extinction  times  less  than 
or  equal  to  e*_i  and  messages  e\  •  •  •  e*_2  (if  k  >  2)  are  all  lost  under  both  policies  as  none  of  them 
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could  commence  transmission.  Message  e*_i  is  lost  under  x  as  it  could  not  begin  transmission  and 
under  ft  as  it  could  not  complete  transmission.  At  time  e*_i,  schedule  ek  under  ft  and  assign  the 
residual  service  time  of  ek  under  x  to  be  equal  to  the  new  service  time  of  ek  under  ft.  Again,  this 
becomes  possible  because  service  times  are  independent  and  exponentially  distributed  with  the  same 
rate.  Also,  let  ft  follow  x  hence  onwards;  clearly  this  leads  to  the  conclusion  that  LJ  =  L*  Vt  >  t0. 

Case  2  :  a  <  ek-i . 

At  time  <r,  the  sets  of  extinction  times  under  the  two  policies  differ  only  in  that  e^-i  is  included 
in  that  set  under  x  as  compared  to  ek  under  ft.  Let  ft  follow  x  after  a  except  that  it  schedules  e* 
when  (and  if)  x  schedules  ej,_i.  Thus  ft  is  well  defined  in  [<r,  r),  where  r  is  the  end  of  the  current 
busy  period  under  x.  We  now  have  three  cases  to  consider  depending  on  whether  x  scheduled  e^-i 
or  not,  and  whether,  when  scheduled,  the  service  completed  or  not.  In  what  follows,  we  treat  only 
the  case  in  which  x  never  managed  to  schedule  ek- j.  Other  cases  can  be  handled  similarly  and  we 
omit  their  discussion. 

Suppose  x  never  scheduled  e*_i  for  service  in  [<r,  e*- 1).  Clearly  r  >  ejt_x  as  x  is  nonidling.  If 
r  >  e*  as  well,  letting  ft  follow  x  after  time  r  ,  it  follows  that, 

=  L*  +  l{ek-i  <t<e  k]  Wt>  t0. 

Suppose  now  that  r  <  ek.  Schedule  ek  under  x  at  time  r.  If  there  are  no  arrivals  while  ek  is  in 
service  under  ft,  we  let  ft  follow  x  from  time  ek  onwards  and  conclude 

L*  =  Xf  +  l{efc_i  <  t  <  ek}  +  l{t  >  ek,a\  <  ek}, 


where  <7X  is  the  time  at  which  e*  leaves  the  system  under  ft. 

Consider  now  the  case  in  which  there  is  at  least  one  arrival  in  [r,  a{)  and  the  first  message  that 
starts  service  under  x  has  extinction  time  et,.  Assign  the  service  of  e&  under  x  to  be  equal  to  the 
residual  service  time  of  ek  under  ft.  Let  a2  be  the  instant  at  which  the  above  service  would  have 
completed  if  uninterrupted.  Consequently  we  have  to  discuss  the  following  cases: 

1)  Suppose  ek  <  eb  and  a2  >  ek.  Then  upon  scheduling  eb  under  ft  at  time  ek  and  assigning  the 
remaining  service  time  under  x  to  be  equal  to  the  new  service  time  of  eb  under  ft,  and  letting  ft 
follow  x  hence  onwards,  we  obtain, 


f  L\  if  t  6  [to,e*-i)U[e*,oo), 

\Lf  +  1  if*  G  [e*_i,e*). 


13 


2)  Suppose  now  that  ej,  <  e*  and  o2  >  e^.  Then,  at  time  e&,  the  message  ej,  is  lost  under  both 
policies.  If  no  messages  begin  service  under  x  in  [et,  #2),  letting  x  follow  x  from  time  o2  onwards, 
we  have, 

Zf  =  L\  +  l{e*_i  <  f  <  e*}  +  l{t  >  <  e*,}. 

On  the  other  hand,  suppose  a  message  begins  service  under  x  in  [eb,  cr2).  This  is  a  situation  we  have 
already  discussed.  By  repeating  the  arguments  in  an  obvious  way,  we  are  led  to  Zf  >  L*  'it  >  t0. 

3)  Consider  the  case  <r2  <  min(e(,,e*).  Let  x  follow  x  for  t  >  <r2  and  assume  again  that  the 
corresponding  services  are  of  equal  duration.  Thus  x  is  well  defined  in  [cr2,  ri),  where  now  rx  is  the 
end  of  this  current  busy  period  under  x.  If  T\  >  e^,  one  has 


/  L\  if  t  €  [t0,  U  [et,  00); 

\  Lf  +  1  if  t  6  [efc-i.ej,). 


For  T\  <  eb,  schedule  e;,  under  x  and  proceed  as  discussed  before  to  obtain  Zf  >  Zf  Vt  >  to- 

4)  If  eb  =  €k  and  <r2  >  Zb  —  e*  ,  letting  x  follow  x  from  time  e;,  =  efc  onwards,  one  has  Z*  = 
Zf  Vi  >  i0- 

Remarks. 

1.  The  theorem  is  not  true  without  the  exponential  service  assumption. 

Counterexample:  Consider  a  single  server  queueing  system  in  which  two  messages  with  extinction 
times  2  and  6  units  are  initially  present  in  the  system.  The  service  times  form  a  renewal  process 
with  the  common  distribution  :  P(S\  =  0)  =  p,  P(Si  =  5)  =  q  =  1  -p.  Assume  also  that  0  <  p  <  q. 
Let  LTE  be  the  policy  which  serves  the  message  with  the  largest  time  to  extinction.  Elementary 
calculations  show  that  for  all  t  >  6, 


with  prob.  pq\ 
with  prob.  p2  +  pq; 
with  prob.  q2. 


where  Llte  is  obtained  by  coupling  the  service  times  to  those  for  Lste.  Since  Llte  has  the  same  law 
as  L,te,  it  follows  that  for  all  t  >  6, 

E[L‘te]  -  E[L lte]  =  pq-q2  =  (p-  q)q  <  0. 


2.  Consider  now  the  situation,  as  in  Remark  2  following  Theorem  II.  1,  of  a  single  server  queue 
under  assumption  A1  when  arrivals  into  the  system  are  disallowed  after  the  system  has  started 
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operation.  Following  the  proof  of  Theorem  III.  1 ,  it  is  clear  that  under  the  additional  assumption  of 
exponential  service  times,  STE  is  optimal  in  the  class  Tj  of  nonpreemptive  policies.  The  example 
in  the  previous  remark  shows  the  necessity  of  this  additional  assumption.  This  is  in  contrast  to  the 
result  in  section  II. 

We  now  consider  optimality  within  Tx,  the  broader  class  of  nonpreemptive  policies  that  permit 
idling.  A  careful  examination  of  the  previous  example  shows  that  with  only  assumption  Al,  the 
optimal  policy  need  not  belong  to  the  class  of  STEI  policies.  Also,  a  slight  modification  of  the 
above  example  shows  that  the  STE  policy  need  not  be  optimal  under  assumptions  Al  and  A2. 
These  observations  should  be  compared  to  the  conclusions  of  Proposition  II.  1  and  its  Corollary. 
However,  we  can  prove  the  following: 

Proposition  III.l  For  a  single  server  queue  under  assumptions  Al  and  exponential  service  time 
distributions ,  for  every  x  E  Tx,  there  exists  it  E  STEI  such  that  for  each  initial  state  z  E  Z, 
L*{z)  <st  L*{z). 

Corollary  III.l  For  a  single  server  queue  as  above  and  the  added  assumption  A2,  we  have, 
Late(z )  <,t  L*(z)  Vtt  6  Tx,  V*  E  Z. 

The  proofs  are  almost  verbatim  versions  of  those  of  Proposition  II.  1  and  its  Corollary  with 
minor  modifications  and  hence  are  omitted. 

Let  us  now  consider  policies  that  are  allowed  to  preempt  messages  if  necessary,  and  consider 
optimality  within  T,  the  class  of  arbitrary  (nonanticipative)  policies.  Let  STE(P)  denote  a  pre¬ 
emptive  version  of  the  STE  policy,  that  is,  a  policy  in  T  which  always  transmits  the  message  with 
the  shortest  time  to  extinction;  it  interrupts  current  transmission  when  there  is  an  arrival  with  an 
earlier  extinction  time  and  commences  transmission  of  this  message. 

We  have  the  following  result  : 

Theorem  III.2  Consider  a  single  server  queue  under  assumption  Al.  Assume  also  that  the 
common  distribution  of  the  service  times  is  exponential.  Then,  for  each  initial  state  z  E  Z,  we  have 

jr*te(p)(z)  <4t  Lir V7r  e  T 

Proof.  To  avoid  repetitiveness,  we  will  only  outline  the  proof.  Consider  any  policy  x  E  T.  Let 
E(to)  =  {ex  •  •  •  en}  where  to  is  an  arbitrary  decision  instant.  Assume  that  at  to,  either  n  >  2  and 
7r  schedules  e*,  (2  <  k  <  n)  or  n  >  1  and  x  chooses  to  idle  .  We  will  construct  f  which  schedules 
ex  at  time  to,  takes  the  next  action  only  at  an  event  epoch  (i.e.  at  an  instant  of  an  arrival,  service 
completion  or  the  loss  of  a  message)  and  satisfies  L*  <,*  Lr .  Usual  induction  arguments  then 
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complete  the  proof. 

Consider  the  parallel  evolution  of  the  system  under  the  two  policies  x  and  x  and  couple  the 
realizations  by  giving  them  the  same  arrival  and  deadline  processes. 

Consider  the  case  when  n  >  2  and  x  schedules  e*  (2  <  k  <  n). Depending  on  the  nature  of  the 
next  event,  the  following  cases  have  to  be  discussed: 

(a)  The  next  event  is  a  service  completion  (under  both  x  and  x).  Let  x  follow  x  henceforth,  except 
that  (i)  it  schedules  e*  when  (and  if)  x  schedules  ei,  and  («)  it  either  never  schedules  or  preempts 
and  never  reschedules  e*  if  e\  is  lost  under  x.  The  usual  arguments  then  lead  to  the  relation 
If  <  LJ  it>  t0. 


(6)  The  next  event  is  either  the  loss  or  the  arrival  of  a  message  (under  x  and  x).  Letting  x  to  be 
identical  to  x  hence  onwards,  we  have  L\  —  L*  it  >  to. 

(c)  Suppose  that  x  simply  preempts  e*  without  any  other  event  happening.  If  x  schedules  ex,  define 
x  from  this  instant  onwards  to  follow  x.  Otherwise,  we  are  in  a  situation  similar  to  that  at  time 
to,  and  the  arguments  can  be  repeated. 

Consider  now  the  case  when  n  >  1  and  x  decides  to  idle.  The  important  point  here  is  that  the 
because  of  the  coupling,  the  first  time  x  decides  to  schedule  a  message  can  be  determined  following 
the  realizations  of  x.  The  proper  actions  of  x  can  then  be  stipulated  to  lead  to  the  desired  result. 
The  accounting  is  straight  forward  but  very  repetitive  and  hence  is  omitted. 

The  proof  is  then  completed  by  noting  that  L*  was  chosen  to  have  the  same  distribution  as 


Lt- 

Remarks. 

1.  The  counterexample  following  Theorem  III.l  demonstrates  the  necessity  of  the  assumption  of 
exponential  service  time  distribution. 

2.  From  the  proof  it  is  clear  that  idling  never  pays  under  assumption  A1  when  premption  is  allowed. 

3.  Let  T*(z)  =  {T*  (z),  k  =  1, 2,  •  •  •}  where  T£(z)  d=f  inf{t  >  0  :  LJ(z)  >  &}.  for  k  =  1, 2,  •  •  •  be  the 
discrete  time  process  representing  the  instants  at  which  messages  are  lost  under  a  policy  x  when 
starting  from  state  z  at  time  0.  Since 


T"i{z)>utT**(z)  <=>  L"'(z)<.tL**(z), 


for  any  two  policies  xi  and  X2,  all  results  in  Sections  II  and  III  are  valid  for  the  cost  T£  with  the 
obvious  modifications.  In  reliability  applications  [2],  the  system  is  often  considered  to  have  failed 
when  the  first  deadline  is  missed.  The  problem  of  stochastically  maximizing  the  RV  T{(z)  is  then 
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appropriate.  It  is  interesting  to  observe  that  for  the  optimality  of  STE  in  this  case,  exponentiaJity 
of  service  times  is  not  needed  in  Theorem  II.  1  and  in  all  the  results  of  this  section. 

Weaker  versions  of  some  of  the  results  above  were  presented  in  [4].  In  that  paper,  optimality 
with  respect  to  the  long  run  expected  number  of  successful  customers  was  considered.  A  theorem 
analogous  to  our  Theorem  III.l  was  proven  when  arrivals  are  Poisson  and  optimality  is  considered 
within  the  class  of  stationary  policies.  Statements  analogous  to  our  Proposition  III.l  and  Theo¬ 
rem  III.2  are  also  given  under  an  additional  hypothesis  (Assumption  4.2  in  [4],  Page  71),  which 
seems  difficult  to  verify.  Furthermore,  the  arguments  and  methodology  used  there  are  substantially 
different  from  ours. 

IV.  SCHEDULING  UNDER  A  REDUCED  INFORMATION  STRUCTURE 

In  this  part,  we  re-examine  the  problem  considered  in  the  last  two  sections.  We  assume 
that  due  to  implementations!  difficulties,  the  scheduler  does  not  have  precise  knowledge  of  the 
extinction  times  of  the  messages.  The  information  regarding  the  distribution  of  the  deadlines  is 
however  available  and  there  is  a  mechanism  which  informs  the  scheduler  when  messages  are  lost. 
Furthermore,  the  extinction  times  become  available  as  soon  as  a  message  begins  service  and  actions 
can  be  taken  as  in  the  previous  sections.  Interestingly  enough,  providing  the  scheduler  with  less 
detailed  information  tends  to  simplify  the  situation.  Under  some  reasonable  assumptions  on  the 
deadlines,  idling  policies  and  the  exponentiality  of  service  times  play  a  less  important  role  in  the 
analysis  and  reasonably  complete  results  are  obtained. 

Consider  the  following  single  server  queueing  system.  Customers  arrive  at  times  {t<}  and  depart 
at  times  {<<  +  d<}  if  service  doesn’t  begin  (or  isn’t  completed).  Only  the  common  distribution  F(.) 
of  {di}  is  available  to  the  scheduler,  but  the  scheduler  is  also  aware  of  the  departures  of  customers 
when  they  occur.  Assume  that  F(.)  has  a  density  /(.);  it  is  well  known  that  the  failure  rate 

function  A (t)  =f  f(t)/(  1  —  F(t))  (defined  for  all  those  values  of  t  for  which  F(t )  <  1)  has  a  useful 
probabilistic  interpretation;  namely  A (t)dt  is  the  probability  that  the  RV  having  d.f  F{.)  takes  value 
in  [<,  t  +  dt]  given  that  it  is  greater  than  t.  It  seems  natural  to  assume  that  beyond  a  certain  time, 
the  impatience  of  the  customers  doesn’t  decrease.  We,  therefore,  make  the  following  assumption  : 

(A3)  {d,}^!  is  a  sequence  of  i.i.d  RV’s  which  are  independent  of  {Ti}^.  Furthermore  the  common 

distribution  has  a  non-decreasing  failure  rate  function. 

We  first  consider  the  situation  when  deadlines  are  to  the  beginning  of  service.  A  customer  is 
termed  eligible  at  time  t  if  it  has  arrived  but  neither  begun  service  nor  reneged  by  t.  Let  W(t) 
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denote  the  (decreasing)  ordered  set  of  waiting  times  of  the  eligible  customers  at  time  t.  In  Renewal 
Theoretic  terminology,  W(t )  is  the  ordered  set  of  their  ages.With  Ha(t),  Ct(t),  and  i(t)  as  defined 
in  Section  II,  z(t )  =  (W(t),Ha(t),Ct(t),i(t))  is  a  suitable  state  description  under  assumptions  .41 
and  A3.  Let  Z  be  its  allowable  range  of  values. 

Let  EA  denote  the  policy  in  To  which  schedules,  at  every  decision  instant,  the  eligible  customer 
with  the  earliest  arrival  instant.  This  differs  from  the  well  known  FCFS  policy  because  of  possible 
non-zero  initial  conditions.  Let  EAI  denote  the  class  of  policies  in  Ti  which  are  allowed  to  idle 
when  customers  are  waiting,  but  which  schedule  according  to  EA  mechanism  when  they  choose 
not  to  idle. 

We  approach  our  main  results  (Theorems  IV.l  and  IV.2)  through  the  following  results.  The 
first  lemma  indicates  the  way  in  which  the  non-decreasing  failure  rate  assumption  on  the  deadline 
distribution  is  going  to  be  used. 

To  state  the  result,  we  introduce  some  notation.  For  a  nonnegative  RV  X,  let  Xt  denote  its 
residual  life  after  t  units,  that  is, 


P(Xt  >z)  =  P{X  >  z  +  t  |  X  >  t). 


Let  be  a  sequence  of  i.i.d  nonnegative  RV’s.  It  is  easy  to  show  that  xj*\xj*\-  •  -Xj^ 

are  independent  (and  identically  distributed  if  t{  =  t  Vi).  If  is  independent  of  a  <7-algebra 

of  events  X,  then  so  is  {X^}"=1.  Furthermore, 


Lemma  IV.l  [14,  Prop.  8.1.3]  If  the  common  distribution  has  nondecreasing  failure  rate,  then  for 
every  t\  >  <2  >  •  •  •  >  tn> 

S>t  -*t3  Sit  •  •  •  Sit -X-iK  . 


The  next  lemma,  in  the  same  spirit  as  Propositions  II.l  and  III.l,  characterizes  a  good  set  of 
scheduling  policies  within  the  class  of  admissible  ones. 

Lemma  IV.2  Consider  a  single  server  queue  under  assumptions  A1  and  AS.  Then  for  every  policy 
7r  6  Fi,  there  exists  a  policy  #  €  EAI,  such  that 

L*(z )  <8t  L*(z)  Vz  G  Z. 
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Proof.  Since  the  arguments  closely  parallel  those  in  the  Section  II,  we  will  only  point  out  the 
essential  differences.  Let  W(to)  =  {uq,  u>2,  -  •  -wn}  ( n  >  2)  and  suppose  that  x  schedules 
(2  <  k  <  n )  at  an  arbitrary  decision  instant  to*  We  first  construct  f  6  Ti  which  schedules  Wk-i  at 
time  to  and  satisfies  L%  <,*  Lv.  Couple  the  realizations  under  x  and  x  so  that 

a)  the  arrivals  are  the  same, 

b)  ej  —  e,- *  Vi  ^  k,k  -  1,  e^_x  <  e ^  a.s.  and  el_x  =at  ej_l5  where  ef  is  the  extinction  time 
of  W{  under  policy  x, 

and  c)  the  deadlines  (and  hence  the  extinction  times)  are  the  same  for  all  arrivals  in  (to,  oo). 

Note  that  the  construction  as  stated  in  b)  is  possible  because  of  Lemma  IV.l  and  the  obser¬ 
vations  stated  before  the  lemma.  The  coupling  for  service  times  and  the  suitable  definition  for  x 
in  (to,  oo)  can  now  be  given  using  the  ideas  in  Section  II  and  the  rest  of  the  argument  also  follows 
easily. 

We  show  next  that  idling  doesn’t  help.  This  will  then  further  reduce  the  good  set  of  scheduling 
strategies  to  the  EA  strategy.  Towards  this  end,  start  with  a  policy  x  G  EAI  and  let  to  be  an 
instant  at  which  x  idles.  With  W(to)  =  {uq,  •  •  *icn}  (n  >  1),  let  x  be  another  policy  which 
schedules  w\  at  time  t0.  Let  r  be  the  the  first  instant  after  t0  at  which  x  schedules  a  customer 
and  a  be  the  time  at  which  w\  completes  service  under  it.  Couple  the  realizations  under  x  and  x 
to  have  the  same  arrival  and  deadline  processes.  Note  that  r  can  then  be  determined  observing 
the  realization  of  x.  Now,  if  r  >  <r,  letting  x  follow  x  from  a  onwards  (by  inserting  idle  periods  if 
necessary),  one  concludes,  Vt  >  to,  that 

It  =  L  *  +  l(f  >  ei,ei  <  r). 

When  t  <  a,  the  argument  proceeds  as  follows.  Let  x  begin  serving  ip*  (with  extinction  time  e*) 
at  r.  Take  the  first  service  time  under  x  and  x  to  be  equal.  Also,  let  x  idle  in  [a,  <r  +  r],  and  follow 
x  from  ( a  +  r)  onwards,  except  that  it  never  serves  w*  in  [a  +  r,  oo).  Recalling  that  x  G  EAI ,  it 
follows  that,  Vt  >  to, 

Lj  =  L  *  +  l(ei  <t<e*,wi^  w*). 

We  have  thus  shown  the  following 

Theorem  IV.l  For  a  single  server  queue  under  assumptions  A1  and  AS, 

Lea(z)<,tL*(z)  Vxer  uVzez. 
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When  deadlines  are  to  the  completion  of  service,  the  following  can  be  easily  shown  combining 
the  ideas  given  so  far. 

Theorem  IV.2  Consider  a  single  server  queue  under  assumptions  Al  and  AS.  Assume  further 
that  the  common  distribution  of  the  service  times  is  exponential.  Then 

Lea(z)<tiL*(z)  VttGT i,V*€Z. 


Finally,  we  observe  that  in  sections  II,  III  and  IV,  one  could  also  have  considered  the  problem 
of  maximizing  the  number  of  successful  customers.  With  obvious  modifications,  the  optimality 
results  extend  to  this  situation  as  well. 

V.  RESULTS  FOR  A  TANDEM  NETWORK 

In  this  section  we  consider  a  network  of  M  communication  links  in  tandem.  Messages  arrive 
at  times  at  the  first  link  with  corresponding  deadlines  thus  they  carry  extinction 

times  {cj  —  ti  +  which  become  known  to  the  scheduler  upon  arrival.  We  make  the  crucial 

assumption  that  each  message  must  be  transmitted  in  all  of  the  M  tandem  links,  even  if  its  extinction 
time  expires  while  in  transit  or  prior  to  transmission.  Therefore  messages  are  not  lost  and  it  seems 
more  appropriate  to  refer  to  extinction  times  as  due  dates  as  we  will  do  in  the  rest  of  this  section. 

Let  the  various  classes  of  scheduling  policies  be  as  defined  in  section  II.  The  control  action 
involves  deciding  at  appropriate  decision  instants  at  each  of  the  links  which  message  to  transmit, 
if  at  all,  out  of  the  pool  of  currently  available  messages  at  that  link. 

Consider  operating  the  above  system  over  the  time  interval  [0,T].  Let  Cf  be  the  time  at 
which  the  message  arriving  at  time  t,  with  deadline  d;  completes  service,  under  policy  x,  at  the 
last  of  the  M  links  and  departs  from  the  network.  Set  C*  =  T  if  the  message  does  not  depart  by 
time  T.  Let  h  :  M  i-+  M  be  a  continuous  convex  function  with  h(x)  =  0  Va;  <  0.  When  starting 
from  state  z,  the  total  average  cost  incurred  by  operating  the  system  [0,  T]  under  policy  it  is  thus 
KCfiz)  —  e«)]  where  N  =  N{T)  d=f  max{i:t,-  <  T}  is  the  number  of  arrivals  by  time  T. 
We  are  interested  in  characterizing  the  scheduling  policy  which  minimizes  the  above  cost. 

We  make  the  following  assumption  : 

(A4)  The  service  times  at  each  node  form  a  sequence  of  i.i.d  RV’s  that  are  independent  of  each 
other,  of  the  arrivals  at  the  first  node,  and  of  the  deadlines.  Also,  the  arrival  process  to  the 
first  node  is  nonexplosive. 
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We  only  consider  optimality  within  r0,  the  class  of  nonpreemptive  and  nonidling  policies.  Let 
the  Earliest  Due  Date  (EDD)  policy  denote  the  policy  in  To,  which  at  a  decision  instant  at  any 
node  in  the  network  schedules  the  message  with  the  smallest  due  time. 

Our  main  result  is  the  following 

Theorem  V.l.  Consider  the  tandem  network  as  described  above  under  assumption  A4.  Let 
h  :  M>-+  M  be  a  continuous  convex  mapping  with  h(x)  =  0  Vx  <  0.  Then 
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Proof.  The  proof  again  uses  essentially  the  same  ideas  as  in  Sections  II  and  III.  Let  E(t 0)  = 
{ei  •  •  • e„ }  be  the  set  of  due  times  of  the  messages  awaiting  service  at  node  I  (1  <  I  <  M)  , 
and  suppose  that  x  schedules  e*:(2<fc<n)at  time  to,  an  arbitrary  decision  instant  at  that 
node.  We  will  construct  x  which  schedules  e^-i  at  time  to  and  satisfies  Jr(z)  <  Jr(z), where 
Jv(z)  =  .EEjli  h(C?(z)  -  e,)].  Usual  induction  arguments  will  then  complete  the  proof.  We  drop 
z  for  notational  convenience. 

Couple  the  realizations  of  the  system  evolving  under  x  and  x  by  giving  them  the  same  arrival 
and  deadline  processes.  Take  the  service  time  of  e*  at  node  I  under  x  to  be  the  same  as  that  of 
e*_  1  at  node  I  under  x. 

Consider  first  the  case  I  <  M.  Let  x  follow  hence  onward  all  actions  of  x  at  nodes  1,2 ,•••,/ 
except  that  at  node  I,  x  serves  e*_x  while  x  serves  e*.  Consider  the  actions  of  x  at  the  subsequent 
nodes  7  + 1,  •  •  • ,  M.  Let  x  follow  all  actions  of  x  regarding  messages  other  than  ek  and  e^-i .  About 
actions  regarding  ek  and  e*_  1,  consider  the  following  two  subcases: 

1.  Suppose  that  at  the  Jth  node  (7  +  1  <  J  <  M),  x  prefers  to  schedule  ek-\  when  ek  is  waiting 
at  the  same  node.  Let  x  then  follow  x  regarding  messages  e*  and  ejt_i  as  well;  this  will  result  in 
equal  costs  incurred  under  the  two  policies. 

2.  Suppose,  on  the  other  hand,  that  x  prefers  to  schedule  e*  before  ek- 1  at  each  of  the  nodes 
I  +  1  <  J  <  M.  Let  x  serve  ek- 1  when  x  serves  e*  and  viceversa.  First  note  that  Cf  —  Cf  Vi  ^ 

k,  k  —  1  and  U  d=f  Cjl  =  C£_x  <  V  d=f  =  CJ  for  all  sample  paths  and  hence 

N 

A  =  '£WCf-ei)-h(Cf-ei)) 

t=l 

=  h(V  -  ek~i)  +  h(U  -  ek)  -  h(U  -  ek-i)  -  h(V  -  ek). 
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The  assumed  properties  of  h  imply  that  h  is  nonnegative  and  nondecreasing  and  that 

h(x)  +  h{y)  >  h(zi)  +  h(z2), 

for  any  zi,  z2  €  [x,y]  such  that  mm(zi,z2)  —  x  <  y  —  max(zi,22).  This  immediately  shows  that 
A  >  0. 

When  I  =  M,  the  policy  f  is  defined  in  a  similar  fashion;  we  omit  the  details  since  the 
discussion  will  be  verbatim.  Since  C*  has  the  same  distribution  as  C*  for  each  i,  the  conclusion 
follows. 

Remarks. 

The  optimal  scheduling  policy  just  obtained  has  a  nice  distributed  implementation;  actions  which 
are  globally  optimal  can  be  taken  at  each  queue  based  just  on  the  due  times  of  the  messages  waiting 
at  that  queue  only. 


VI.  CONCLUSIONS 

We  have  considered  the  problem  of  dynamically  scheduling  the  transmission  of  messages  in 
situations  where  messages  have  individual  constraints  on  either  their  waiting  time  or  complete 
transmission  times.  First,  a  single  node  was  considered  in  which  the  messages  were  lost  if  the 
constraints  were  not  met.  Different  classes  of  admissible  policies  were  considered.  Under  nonex¬ 
plosive,  but  otherwise  arbitrary,  arrival  and  completely  arbitrary  deadline  processes,  but  under 
some  restrictions  on  service  times,  the  intuitively  appealing  policy  of  scheduling  the  “most  urgent” 
message  was  shown,  in  many  cases,  to  minimize  the  number  of  messages  lost  in  a  strong  pathwise 
sense.  A  tandem  network  of  M  nodes  was  then  considered.  We  assumed  that  messages  are  never 
lost,  but  a  penalty  is  incurred  if  a  message  doesn’t  complete  service  in  the  tandem  by  its  extinction 
time  (also  called  due  time  in  this  case).  Under  fairly  general  restrictions  on  arrival,  deadline  and 
service  processes,  the  intuitive  policy  of  scheduling,  at  each  node,  the  message  which  is  “earliest 
due”  is  shown  to  minimize  a  tardiness  cost  over  a  finite  operating  horizon  among  all  nonidling, 
nonpreemptive  policies.  Simple  interchange  arguments  were  used  to  establish  all  our  results.  As  is 
clear  in  the  development  of  the  paper,  the  particular  application  on  which  we  concentrated  is  only 
incidental  in  that  the  entire  approach  (model  formulation  and  solution)  is  quite  general  and  applies 
to  abstract  queueing  systems  with  customers  and  services  of  unspecified  application  features. 
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